Compact Transformer Tracker with Correlative Masked Modeling

Transformer framework has been showing superior performances in visual object tracking for its great strength information aggregation across the template and search image with well-known attention mechanism. Most recent advances focus on exploring mechanism variants better aggregation. We find these schemes are equivalent to or even just a subset of basic self-attention In this paper, we prove that vanilla structure is sufficient aggregation, structural adaption unnecessary. The key not structure, but how extract discriminative feature enhance communication between target image. Based finding, adopt vision transformer (ViT) architecture as our main tracker concatenate embedding. To guide encoder capture invariant tracking, attach lightweight correlative masked decoder which reconstructs original from corresponding tokens. serves plugin compact skipped inference. Our uses most simple only consists ViT backbone box head, can run at 40 fps. Extensive experiments show proposed transform outperforms existing approaches, including advanced variants, demonstrates sufficiency tasks. method achieves state-of-the-art performance five challenging datasets, along VOT2020, UAV123, LaSOT, TrackingNet, GOT-10k benchmarks. project available https://github.com/HUSTDML/CTTrack.